Shifting ink images overlaid over text images in email documents

ABSTRACT

In a document in which a plurality of data items of different kinds are mixed, when one data item is edited, the relative positional relation to other data items is prevented from being destroyed, whereby information is prevented from becoming meaningless or from being changed. For example, when an edit is carried out on one data item, a deviation amount of that data item is derived and a shift process by the same amount is effected on the other data items, whereby the relative positional relation can be maintained among the data items.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the technology for displaying bothimage information and locus information.

The present invention relates to the technology for editing electronicmail data.

2. Related Background Art

There were conventional devices to display a plurality of informationpieces of different kinds in an overlapping state, but they had theproblem that when one of the information pieces displayed in theoverlapping state was edited, output positions of the other informationpieces were not changed and thus the relative positional relation amongthe plurality of information pieces differed between before and afterthe edit.

When the relative positional relation was desired to be maintained, theinformation to specify the output positions of the other informationpieces must be input in one information piece and this resulted indrawbacks of complicated information and heavy edit processing ofinformation.

SUMMARY OF THE INVENTION

The present invention actualizes such technology that in a state where aplurality of data items of different kinds are displayed in a mixedstate, when an edit process is effected on one data item, a deviationamount of data due to the edit is calculated and a shift process by thesame amount is also effected on the other data items, whereby therelative positional relation is maintained among the data items.

The present invention also actualizes such technology that when anoperator receives an electronic mail including data of image etc. inaddition to a text and responds to or forwards this received mail,consideration is given to insertion of a quotation character string orthe like into the text part, a deviation amount of the text defined by aformat of this character string inserted is calculated, and a shiftoperation by the same amount is also effected on the image, whereby thedata of the reply mail or the forwarded mail can maintain the data ofthe same contents as the received data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram to show the schematic structure of an apparatusaccording to the present invention;

FIG. 2 is a diagram to show the appearance of an apparatus according tothe present invention;

FIG. 3 is a diagram to show the functional structure of a systemaccording to the present invention;

FIG. 4 is a diagram to show an example of a document prepared in anembodiment of the present invention;

FIG. 5 is a flowchart of the whole processing flow for preparing thedocument;

FIG. 6 which is comprised of FIGS. 6A and 6B is a flowchart ofprocessing in a record pen mode;

FIG. 7 is a diagram to show an example of a document preparation windowin an edit mode;

FIG. 8 is a diagram to show an example of a pen menu in the edit mode;

FIG. 9 is a screen upon setting of attributes of a record pen;

FIG. 10 is a diagram to show an example of the screen before input of adynamic stroke with the record pen;

FIG. 11 is a diagram to show an example of the screen in the middle ofinput of a dynamic stroke;

FIG. 12 is a diagram to show an example of the screen after completionof data input of one object;

FIG. 13 is a diagram to show data structures;

FIG. 14 is a flowchart of a reproduction process of object;

FIG. 15 which is comprised of FIGS. 15A and 15B is a flowchart of areproduction routine;

FIG. 16 which is comprised of FIGS. 16A and 16B is a flowchart of astroke reproduction process;

FIG. 17 is a diagram to explain the screen in the middle of reproductionof the document of FIG. 4;

FIG. 18 is a diagram to explain the screen in an object-selected state;

FIG. 19 is a flowchart of a movement process of object;

FIG. 20 is a drawing to show an example of the screen in a state inwhich an object to be moved is designated;

FIG. 21 is a drawing to show an example of the screen in the middle ofmovement of the object;

FIG. 22 is a diagram to show an example of the screen at the end ofmovement of the object;

FIG. 23 which is comprised of FIGS. 23A and 23B is a flowchart of aneraser process;

FIG. 24 is a diagram to show an example of the screen in an eraser mode;

FIG. 25 is a diagram to explain circumscribed rectangles aroundrespective strokes as grouped;

FIG. 26 is a diagram to show an example of the screen where a stroke iserased by an eraser;

FIG. 27 is a flowchart of a quotation process;

FIG. 28 is a diagram to show an example of a received mail; and

FIG. 29 is a diagram to show an example of a reply mail.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments of the present invention will be described in detail byreference to the drawings.

FIG. 1 is a block diagram to show the schematic structure of anembodiment of the present invention.

Numeral 10 designates a CPU (central processing unit) for controllingthe whole of this apparatus, which controls execution of variousprocesses associated with the present invention, according to controlprograms stored in memory 14, hard disk 16, floppy disk 18, and so on.

Numeral 12 denotes a peripheral I/O circuit for memory control and I/Ocontrol.

Numeral 14 represents a memory (MEM) comprised of ROM and RAM, whichstores the control programs for the processes associated with thepresent invention, which will be described hereinafter with theflowcharts, and various parameters used for the respective processes andwhich is also used as a work area for storing data appearing in eachprocess.

Numeral 16 indicates a hard disk, which stores the control programs ofthe processes associated with the present invention, which will bedescribed hereinafter with the flowcharts, and the various parametersused for the respective processes.

Numeral 18 stands for a floppy disk, which stores the control programsof the processes associated with the present invention, which will bedescribed hereinafter with the flowcharts, and the various parametersused for the respective processes. These data may be directly used ormay be used after downloaded in the hard disk 16 or in the memory 14.Without having to be limited to the floppy disk, the memory may beanother detachably mounted storage medium to this apparatus, forexample, such as a CD-ROM or a magneto-optical disk.

Numeral 20 is a mouse, which is used for designating coordinates of adesired position on a display screen and also for effecting variousselect operations by clicking on its button.

Numeral 22 denotes a keyboard, and by depressing keys thereof, variouscharacters, symbols, functions, etc. corresponding to the keys can beinput, thereby permitting input or editing of text. This keyboard may bea soft keyboard displayed on the display screen, without having to belimited to the key input device.

Numeral 24 represents a digitizer, through which coordinates of adesired position on the display screen can be input by moving a pen (44in FIG. 2) on this digitizer. The present embodiment will be describedin the form of an example in which the digitizer 24 is overlaid on thedisplay device 26, as shown in FIG. 2, whereby information of anarbitrary point, a trace, or the like can be input as if to overlay iton an image displayed in the display device 24, but this coordinateinput means is not limited to the example of the overlaid configurationon the display device. For example, the coordinate input means may be ofa tablet type, in which the input device is placed on a desk separatelyfrom the display device and in which the coordinate information of apoint, a trace, or the like can be input through the coordinate inputdevice on the desk with observing the screen. If only rough data ishandled, the coordinate input means can be a touch panel of a pressuresensitive type for designating an arbitrary point by finger.

Numeral 26 indicates the display device such as a liquid crystal displaypanel or a CRT, and 28 a display control circuit for making the displaydevice 26 display a desired image under control of the CPU 10.

Numeral 30 denotes a voice input/output device composed of a microphoneand a loudspeaker (for example, a handset), and 32 a voice controlcircuit for controlling input/output of voice through the voiceinput/output device 30 under control of the CPU 10. The voiceinput/output device 30 is not limited only to the integral form of themicrophone and loudspeaker, but it may also be a microphone and aspeaker separately provided.

Numeral 34 designates an image input/output device for facsimiletransmission (i.e., including an image scanner and a printer), and 36 aFAX control circuit for controlling the image input/output device 34under control of the CPU 10. The image input/output device 34 is notlimited only to the integral device incorporating the image scanner asan image input device and the printer as an image output device, but itmay also be the image input device and the image output deviceseparately provided.

Numeral 38 indicates a modulator/demodulator (modem) forfrequency-modulating a signal to be transmitted to a frequency bandsuitable for a telecommunication line and for demodulating a modulatedsignal from the telecommunication line, and 40 a communication controlcircuit for controlling connection and communication with thetelecommunication line, including establishment of a communicationprotocol, under control of the CPU 10. The telecommunication line can bea line used for transmission of information, such as a public line orLAN. Numeral 39 stands for a data bus for exchange of data between thecomponents.

FIG. 2 is a front elevation to show the appearance of the device in thepresent embodiment. The display device 26 and digitizer 24 are placed inthe front and the handset 30 on the left side. A ten key 42 for input oftelephone number is located on the right side. Numeral 44 designates aninput pen of the digitizer 24.

FIG. 3 is a diagram to show the functional structure of a systemaccording to the present embodiment, and the functions are carried outby the components disclosed in FIG. 1.

Image input/creation means 50 outputs to input control/memory means 56data of an image read through the image input/output device 34 or animage of a document including a text created using the keyboard etc. bywordprocessor or plotter software to operate on this apparatus.Coordinate input means 52 is the digitizer 24, which calibrates andtimes the coordinates input through the pen 44 to output correctedcoordinate values to the input control/memory means 56. Voice inputmeans 54 performs analog-digital conversion of a voice signal inputthrough the handset 30 to output a digital signal to the inputcontrol/memory means 56. The outputs from these means 50, 52, 54 aresupplied to the input control/memory means 56 to be stored as data.

When a message by voice and/or locus is added to the input image, theinput control/memory means 56 makes display means 58 (i.e., the displaydevice 26) display the input image, the details of which will bedescribed hereinafter. While observing the display screen, the userinputs an arbitrary locus through the digitizer 24 or through the mouse20 and/or inputs the voice through the handset 30.

Each data of the image, voice, and locus input and stored in the MEM 14is transmitted through the public line under control of communicationcontrol means 62.

Communication control means 64 on the reception side receives theinformation through the public line and writes the received data inmemory means 66. In response to a disclosure request from the user onthe reception side, output control means 68 reads the data of receivedimage specified by the user from the memory means 66 and supplies thedata to image output means 72 (the printer section of the imageinput/output device 34) and/or to display means 74 (the display device26 and display control means 68). When the data is accompanied byvoice/locus data, the output control means 68 first supplies thereceived image to the display means to make the display means displaythe received image and also supplies the locus data to the display means74 and/or the voice data to voice output means 76 in accordance with areproduction instruction of voice/locus given by the user. This causesthe locus input on the transmission side to be displayed at the sameposition and at the same speed as those of the input locus on thetransmission side so as to be overlaid on the received image displayedon the display means 74. In synchronism with this, the voice isreproduced at the same timing as that of the voice input together withthe locus. For printing the received image on paper, the image outputmeans 72 is used.

Although in FIG. 3 the functions of input and output of image,coordinates, voice, and text are illustrated separately on thetransmission side and on the reception side, the apparatus in thepresent embodiment is ready for both transmission and reception as shownin FIG. 1 and FIG. 2, so that the same apparatus can perform both inputand output of image, coordinates, voice, and text. Accordingly, afterinputting and editing the image, coordinates, voice, and/or text on thetransmission side, the user can check the data by reproducing the image,locus, voice, and/or text having been entered, by use of the displaydevice, the handset, or the image output device; on the reception sideto receive the data, the user can not only reproduce the received data,but also input an additional image, coordinates, voice, and/or text inthe received data or edit the received data by use of the mouse,keyboard, digitizer, handset, or image input device.

The processes of the present embodiment will be described in detail withexamples of flowcharts and display screens in respective states.

FIG. 4 illustrates an example of a document prepared for explaining theway to a hospital, in which in a document window 410 there are displayeda text 401 of explanation entered through the keyboard 22, “The way tothe hospital”; a text 402 entered through the keyboard 202, to describethe telephone number and consultation hours; an image 403 of a maparound the hospital, entered through the scanner section of the imageinput/output device 34 or from another terminal equipment through thepublic line; loci 404 to 406 entered through the mouse 20 or through thedigitizer 24; and icons 407 to 409 to indicate entries of dynamic locusinformation and voice and to enable to designate reproduction of adynamic locus or sound by clicking either one of them by the mouse 20 orby the digitizer 24. The “clicking” means depressing the button of themouse 20 or dropping the pen down onto the digitizer 24, and it isdetermined according to the clicking that the user selected theinformation on the display screen corresponding to the coordinates ofthe position of the mouse cursor at that time or the coordinates of theposition of pen down, for example an object displayed at that position.

The dynamic stroke herein, though will be detailed hereinafter, is astroke that holds not only the coordinate information of stroke but alsotime information about input time of each coordinate information andthat thus enables reproduction of a locus at the as-input speedaccording to the time information.

By turning the microphone (voice input means) on upon entry of thedynamic stroke, an explanation by voice can also be entered in additionto the coordinate information entered by the locus. For example, uponentry of the locus 405 of FIG. 4, the user makes a voice of “thisjunction of five roads” at the time when the locus reach the junction offive roads and also makes a voice of “turn to the right at the corner ofthe bookstore” at the time of input of the locus turning at the cornerof the bookstore, which enables to input information easier tounderstand.

When complete reproduction of the dynamic loci and voice is designatedto the document thus prepared (FIG. 4), (1) the locus 405 and locus 406are reproduced as they were drawn, together with the voice of “Turn tothe left at an acute angle at this junction of five roads, turn to theright at the corner of the bookstore that is seen right, and find thehospital as the third building on the left side. You will take aboutfour minutes on foot from the station to the hospital.”; (2) the locus404 is reproduced as it was drawn; and (3) the voice of “Yourreservation time is eleven o'clock and please be on time” is reproduced.

The process for preparing this document will be described according tothe flowchart of FIG. 5.

The preparation of document is carried out in the document window 410displayed on the display screen of the display device 26. When thedocument window is opened, a parameter to indicate a current mode storedin the MEM 14 is initialized (S2). Occurrence of an event is detected bywhether some information is intended to be entered through the inputdevice, i.e., through the mouse 20, keyboard 22, digitizer 24,microphone 30, or image input means 34 (S4), and thereafter a kind ofthe event detected will be determined in S6 to S10. If in S6 the eventis determined to be an end event, for example, by clicking on the endbutton on the display screen through the mouse 20 or the digitizer 24,this operation will be terminated. If in S8 the event is determined tobe an entry of character code from the keyboard 22, the process willmove to a character input routine of S26. In this character inputroutine, characters entered from the keyboard 22 are successivelydisplayed in the form pursuant to the document format stored in thememory 14, at the position of the cursor 702 for character inputdisplayed on the document preparation window 410 of FIG. 7. A charactergroup entered through the keyboard 22, i.e., text data is stored in theMEM 14.

When in S10 the event is determined to be an instruction of mode switchgiven by clicking a button in a pen menu 701 displayed in the documentpreparation window 410 through the mouse 20 or through the digitizer 24,the parameter to indicate the current mode in the MEM 14 is reset to adesignated mode (S12) and a menu screen established for that mode isdisplayed (S14).

The pen menu 701 displayed in the edit mode is illustrated in FIG. 8,and each of the buttons will be described. This pen menu can be moved toan arbitrary position and displayed there by dragging a title bar, as anormal dialog box can. The MEM 14 always holds the coordinateinformation to indicate the displayed position of the pen menu and thepositions of the respective buttons, and on the occasion ofdetermination of the kind of the mode designated in S10, this coordinateinformation stored in the MEM 14 is compared with the coordinateinformation of the position clicked by the mouse 20 or by the digitizer24, whereby a mode corresponding to the button at the click position isdetermined to be the designated mode.

The button 801 is a draw mode instruction button for providing aninstruction of switch to a draw mode, the button 802 an edit modeinstruction button for providing an instruction of switch to an editmode, the button 803 an eraser mode instruction button for providing aninstruction of switch to an eraser mode, the button 804 a record peninstruction button for providing an instruction of switch to a recordpen mode, the button 805 an undo instruction button for providing aninstruction for canceling a previous operation and restoring the statusbefore the previous operation, the button 806 a cut button for providingan instruction for cutting data in a range designated, out of thescreen, the button 807 a copy button for providing an instruction forholding data in a range designated, in another area while leaving thedata on the screen, the button 808 a paste button for providing aninstruction for displaying the data in the range specified by the cut orcopy operation at a position designated, the button 809 a clear buttonfor providing an instruction for deleting data in a specified range fromthe screen, the button 810 a smooth button for providing an instructionfor smoothing a coordinate string of locus data designated, the button811 an engross button for providing an instruction for engrossing locusdata specified to a shape designated out of a straight line, a circle(an ellipse or a complete circle), a triangle (normally, an isoscelestriangle or a regular triangle), and a rectangle (a rectangle or asquare), the button 812 a rotate button for providing an instruction forchanging data in a designated range into a rotated display state, thebutton 813 a resize button for providing an instruction for displaying achange instruction button to change the size of data in a designatedrange and for changing the size of the data according to the operationof the button, the button 814 a group button for providing aninstruction for grouping plural items of stroke data designated, thebutton 815 an ungroup button for providing an instruction for ungroupingplural items of stroke data designated into separate independent dataitems not grouped, the button 816 a full select button for providing aninstruction for selecting all data composing one document displayed inthe document preparation window 410, the button 817 a pen set button forproviding an instruction for displaying a screen for changing anattribute of the pen for inputting a locus, i.e., an attribute of thelocus (a color of line and a thickness of line), and the button 818 areproduction start button for providing an instruction for reproducingthe dynamic stroke and voice data. When this reproduction start button818 is clicked, the dynamic stroke and voice data included in thatdocument is reproduced in the input order or plural objects selected arereproduced in a selected order. Numeral 819 indicates a volume window todisplay an image of a swinging needle, thereby indicating the advance ofinput of voice data or the advance of reproduction, and numeral 820 avolume adjust bar to indicate the volume of reproduced voice and topermit the user to drag the bar up and down to change the position,thereby adjusting the reproduction volume of voice.

When the current mode is not the edit mode but either one of the drawmode, the eraser mode, and the record pen mode, the pen menu at thattime is different from the one shown in FIG. 8 but is one like the penmenu 901 shown in FIG. 9, which is composed of the draw mode button 801,the edit mode button 802, the eraser mode button 803, the record startbutton 920 for giving an instruction for starting sound recording, theundo button 805, the volume window 819, and the record pen set button910 for giving an instruction for displaying the screen for changing theattribute of the pen for input of a dynamic locus, i.e., the attributeof the dynamic locus (color of line and thickness of line), and theattribute of voice (volume and presence/absence of compression process).Once the record start button 920 is clicked, a record end button 921 isdisplayed instead of the record start button, however.

When it is determined in S10 that either one of the draw mode button801, the edit mode button 802, the give an instruction of on or off ofcompression of voice. The operator sets arbitrary attributes on thisrecord pen set dialog box 902 by use of the mouse 20 or the digitizer 24and thereafter starts input. The attributes set in the record pen setdialog box 902 are held in the MEM 14 and the attribute data in the MEM14 is also changed every reset of attributes.

FIGS. 6A and 6B illustrate the flowchart of the process in the recordpen mode of S24, and the following explains the process for enteringdata of plural objects comprised of dynamic stroke or voice data. Here,an “object” denotes an object including all data entered during a periodbetween a press of the record start button 920 and a press of the recordend button 921. The icons 407 to 409 of FIG. 4 are displayed for everyobject.

Now, let us explain the input process of dynamic stroke and voice databy the record pen, referring to the flowchart of FIGS. 6A and 6B.

FIG. 10 illustrates the screen resulting from such operations in thecharacter input routine of S26 that the texts 401 and 402 are enteredthrough the keyboard 22, that the image 403 of the map near the hospitalis entered, the image 403 having been entered from the image inputdevice 34 or from another terminal equipment through thetelecommunication line, and that the pen menu 901 is displayed based ondesignation of the record pen mode. Here, the image is handled in thesame level as characters, and by designating input of image in thecharacter input routine, the position to indicate the character cursor702 is displayed as a reference position of the image. Therefore, thedisplay position of image also varies with an edit of character string(text). When the record pen mode is designated, the process shown in theflowchart of FIGS. 6A and 6B starts and it is checked whether therecording device is mounted on the present apparatus (S30). If it isdetermined here that no recording device is mounted on the apparatus,the operator is notified that no voice recording device is mounted, bydisplaying a dialog of “No voice recording device is found. Onlyrecording of handwriting is available.” The screen after this is the oneillustrated in FIG. 10.

After the determination of whether the recording device is mounted ornot in S30, the processor awaits a click on the record start button 920(S32), and with a click it is determined whether the recording device isin an available state (S34). This determination is to determine whetherthe recording device, which was determined to be mounted in S30, is in aready state at the time of the click on the record start button. If therecording device can also be used in a plurality of applications etc.including the present application, the determination of S34 can be madeby providing a flag to indicate whether or not the recording device isin use and checking this flag.

When the recording device is determined to be available in S34, theprocessor checks whether the microphone button (a switch button foron/off of record) is on in the record set area 913 in the record pen setdialog box 902, by reading it from the MEM 14 (S36). When the microphonebutton is on, the record routine 38 is started (S38) and the process ofS38 for recording the voice data in the MEM 14 is carried on until it isdetermined in S42 that an instruction of end is given by a click on therecord end button 921 or until a timeout occurs by a decision that therecording time is over a time preliminarily determined and held as apredetermined time in the MEM 14.

When handwriting data is entered through the mouse 20 or the digitizer24 with the start of the record routine, a routine for recording theinput handwriting in the MEM 14 is carried out repetitively (S40).

When the recording device is determined to be unavailable in S34 or whenthe microphone button is determined to be off in S36, the record routineof S38 is shipped, but the handwriting record routine of S40 isrepetitively carried out before an affirmative decision is made in S42.

When it is determined in S42 that an instruction of end is given by aclick on the record end button 921 or that a timeout occurred over thetime preliminarily determined and held as a predetermined time in theMEM 14, it is determined what kind of data was recorded in S38 and S40(S44), and the program branches to processes in the case of the bothvoice data and stroke data, in the case of only the voice data, and inthe case of only the stroke data. In the example shown in FIG. 4, theobject corresponding to the icon 408 includes the both voice data andstroke data, the object corresponding to the icon 407 includes only thestroke data, and the object corresponding to the icon 409 includes onlythe voice data. It is possible to notify the operator of a kind of dataof each object by using the different types of icons according to thedata kinds of objects.

When the kind of recording is the voice data and stroke data, theprocessor reads a parameter to indicate whether compression isdesignated in the record compression area 914 in the record pen setdialog box 902 out of the MEM 14 (S46). When compression is designated,compression is carried out of the voice data stored in the MEM 14 in S38(S48). When compression is not designated, the processor proceeds toS50.

With the voice data and stroke data (handwriting data) stored in the MEM14 in S38 and S40, the processor performs steps to file the voice data(S50), to prepare a voice cluster (S52), to prepare a stroke clustergroup (S54), to prepare a group cluster (S56), and then to return to theedit mode (S68). FIG. 11 shows the display screen in the middle of inputof the stroke data and voice data before the affirmative decision in S42and FIG. 12 shows the display screen after the affirmative decision inS42. In FIG. 11, the mouse cursor or the cursor to indicate the positiondesignated by the pen is of the shape like 420 during the period betweenthe click on the record start button 920 and the click on the record endbutton 921 and the record end button 921 is displayed in place of therecord start button 920 in the pen menu. During recording of the voicedata, the display is in a state in which the needle of the volume window819 is swinging. In FIG. 12, because the both stroke data and voice dataare entered, the screen displays the icon 408 to indicate presence ofthe both data recorded. The display position of the icon 408 is the leftupper part of the circumscribed rectangle around the stroke data. It is,however, noted that the display position of the icon is not limited tothis position, but the display position of the icon may be the rightupper part of the circumscribed rectangle around the stroke, a positionof coordinates of first input, or any position determined so as not tooverlay the stroke or the image and text below it. When only the voicedata is given without input of stroke data, the icon is displayed at thedefault position.

When the kind of recording is determined to be only the voice data inS44, the processor performs the steps of S58 to S64 and then returns tothe edit mode (S68). The processes of S58 to S64 are the same as thoseof S46 to S52.

When the kind of recording is determined to be only the stroke data inS44, the processor performs the process of S66, which is the same asS54, and then returns to the edit mode (S68).

When it is determined in S44 that no data is recorded, the program alsoreturns to the edit mode (S68).

FIG. 13 shows the structures of the data prepared by the processes ofS50 to S66 and the data entered in the draw mode of S18. These data isstored in the MEM 14.

The cluster data is the data structure stored in the MEM 14 where thestroke data and voice data are grouped to be handled as one object. Forexample, grouping of plural strokes drawn in the draw mode includesautomatic grouping of those strokes where drawing intervals of thestrokes (temporal intervals from pen up to pen down or positionalintervals of the strokes) are smaller than a default value preliminarilydetermined and held as a parameter in the MEM 14, a posteriori groupingwhich is effected on plural items of data selected by the operator atthe time of clicking on the group button 814, and so on.

The normal stroke data is the data structure where one stroke drawn inthe draw mode is stored as single data in the MEM 14.

The reproduction stroke data is the data structure where the data of thestroke entered in the record pen mode, which was expressed previously asa dynamic stroke, is stored in the MEM 14, including not only thecoordinate data of the stroke but also the time information on entry ofeach coordinate position. However, the data of the all dynamic strokesincluded in one object is not held, but only data concerning one stroketherein is held.

The voice data is the data structure where the voice entered in therecord pen mode is stored in the MEM 14.

Each of the cluster data, the normal stroke data, the reproductionstroke data, and the voice data is composed of a portion called a headerincluding Ln (the size of the whole data), Ox (an x-coordinate of theleft upper part of the circumscribed rectangle of data), Oy (ay-coordinate of the left upper part of the circumscribed rectangle ofdata), Cx (a horizontal width of the circumscribed rectangle of data),Cy (a vertical width of the circumscribed rectangle of data), St (aninput start time of data), Et (an input end time of data), and Dk (akind of data stored in the body part); and a portion called a body,including the data itself. In the body part, the normal stroke dataincludes the attribute data concerning the thickness and color ofstroke, and a string of n coordinate points Pi (i=1 to n); thereproduction stroke data includes the attribute data concerning thethickness and color of stroke, a string of n coordinate points Pi (i=1to n), and n pieces of time information Pit (i=1 to n) to indicate thetime of entry of each coordinate point; and the voice data includesszVoiceFileName which is a file name storing the voice data.

An example of cluster data constructed in the layered structure isillustrated as a cluster layered structure. C0H represents the headerpart of the root cluster and C0B the body part of the root cluster, thisC0B including cluster C1 and cluster C4. C1H denotes the header part ofthe cluster C1, C1B the body part of the cluster C1, C4H the header partof the cluster C4, C4B the body part of the cluster C4, and the clusterC1 includes cluster C2 and cluster C3. C2H indicates the header part ofthe cluster C2, C2B the body part of the cluster C2, C3H the header partof the cluster C3, and C3B the body part of the cluster C3.

Depending upon whether data of each cluster is the reproduction strokedata or the voice data, associated data is stored in the header part andin the body part.

Coordinate values of each data stated above are those along thecoordinate axes with the reference at the left upper part of thedocument upon entry of data, and the reference coordinate axes arecoincident with those of the text data.

As described above, the document prepared by the processes describedwith the flowcharts of FIG. 5 and FIGS. 6A and 6B is stored in the MEM14 so that the data of the structures as shown in FIG. 13, together withthe text information, can be identified by a document name as adocument, and thus becomes available for a subsequent process. Forexample, it is possible to print out the document by the image outputmeans 34, to send it through the modem 38 to another facsimile device orcomputer, or to store it in a detachable storage medium such as the FD18 and reproduce it by calling it when necessary.

Next described referring to the flowcharts of FIGS. 14 to 16 areprocesses for reproducing the document after storage or transmission asdescribed above, or for reproducing the document in the middle ofpreparation of document.

Reproduction of the dynamic stroke and voice is started by either of twoways: (1) clicking the reproduction start button 818; (2)double-clicking an icon of an object desired to reproduce. In thereproduction method of (1), objects to be reproduced are switched tothose in either one of the following two cases according to a selectedstate of objects at the time of the click on the reproduction startbutton 818. (1-1) When the reproduction start button 818 is clicked in astate without the select operation, the objects in the document are theobjects to be reproduced and are reproduced in the order of entry of theobjects. (1-2) When some of the objects are clicked into the selectedstate, the objects are reproduced in the order of select operationsthereof.

The flowchart of FIG. 14 will be described herein with the example ofthe process (1-1) where reproduction of objects is designated in thestate without the select operation.

First, the objects as reproduction objects in the document are countedfrom the data in the MEM 14 and the number of objects is stored asnumObj in the MEM 14 (S70). The counter i in the MEM 14 is initializedto 1 (S72), it is determined whether i is larger than numObj (S73), andthe processor goes to S74 with no. The determination of S73 isrepetitively carried out until i is determined to be larger than numObj,i.e., until it is determined that the following processes of S74 to S96are carried out for the all objects determined as the objects in S70.

Since in this example the processes of S74 to S96 are repeated in theorder of entry of the objects, the reproduction objects are retrieved inthe order of entry of the objects (S74). This order of entry of theobjects can be determined by checking St (input start time) in theheader part of cluster data stored in the MEM 14 and retrieving the datain order from the earliest.

The data of the i-th reproduction object is copied onto the working areain the MEM 14 (S76) and the icon of the i-th reproduction object iserased from the display screen (S78). It is determined if the i-thobject includes a reproduction stroke (S80). If there is thereproduction stroke then the stroke is erased from the display screen(S82). Image data (for example, another stroke, text, image, or thelike) located below the stroke is drawn in pixels where the reproductionstroke was erased.

FIG. 17 shows the display screen after the process of S82 in the statewhere the reproduction start button 818 was clicked with displaying thedocument exemplified in FIG. 4 to designate the first drawn object (thestrokes 405 and 406 and voice data) as a reproduction object. The icon408 and the strokes 405, 406 of the first object are erased from thescreen.

Then the processor carries out the reproduction routine of data of thei-th object (S86). This process of the reproduction routine will bedescribed hereinafter referring to the flowchart of FIGS. 15A and 15B.When it is determined in S80 that the i-th object includes noreproduction stroke, only the voice is reproduced in S86 withoutexecuting the stroke erase process of S82.

If there is a reproduction stroke (S90) the reproduction stroke is drawnon the display screen (S92). Since the reproduction process of S86 istemporary reproduction of image, the drawing of stroke is carried outagain to draw the image of the stroke.

The icon of the i-th reproduction object is drawn (S94), i isincremented by one (S96), and the processor returns to S73 andrepetitively carries out reproduction of the all objects set in S70.

In the case of above (1-2) described previously as a case to carry outreproduction of objects, the number of objects in the selected state asreproduction objects is set in numObj in S70, and the objects areretrieved in the order of the select operations in S74. In the case ofabove (2), 1 is set in numObj as the number of reproduction objects inS70.

The reproduction routine of S86 will be described referring to theflowchart of FIGS. 15A and 15B. The kind of the reproduction object isdetermined (S100), and the process is branched according to the bothvoice data and stroke data, only the voice data, or only the strokedata. This type of the object can be determined by analyzing the objectdata copied on the working area in S76 and referencing the Dk (kind)data of the header part in this data.

When the data of the reproduction object includes the both voice dataand stroke data, the processor calculates a difference between the voicestart time (St in the header part of the voice data in the object data)and the stroke start time (the earliest of the header part St of thereproduction stroke data included in the object data) and stores thedifference as diffTime in the MEM 14 (S102).

When it is determined that the voice data is compressed (S104), theprocessor decompresses the data (S106), creates an interrupt eventdetection loop (S108), and starts reproduction of voice (S110).

The interrupt event is, for example, a click on the reproduction endbutton, changeover of input mode, or the like.

The reproduction routine (S112) of the stroke will be describedhereinafter with the flowchart of FIGS. 16A and 16B.

When the reproduction of voice is determined to end (S114), the operatoris notified of the end of the reproduction process, for example, byoutputting such a predetermined sound as “zhang ” (S132). Here, thedetermination of the end of the reproduction process of voice is madebased on determination that the reproduction is completed up to the tailof the data stored in the voice file.

When it is determined in S100 that the kind of the reproduction objectis only the voice data, the processor proceeds to S116 to determinewhether the voice data is compressed (S116). When the voice data iscompressed, the processor decompresses the data (S118), creates aninterrupt event detection loop (S120), and starts reproduction of voice(S122). When the reproduction of voice is determined to end (S124), theoperator is notified of the completion of the reproduction process(S132).

When it is determined in S100 that the kind of the reproduction objectis only the stroke data, the processor goes to S126 to store 0 asdiffTime in the MEM 14, creates an interrupt event detection loop(S128), and reproduce the stroke (S130). The reproduction routine ofstroke will be described hereinafter referring to the flowchart of FIGS.16A and 16B, as well as S112. When the reproduction of stroke iscompleted, the operator is notified of the end of the reproductionprocess (S132).

The reproduction process of stroke is carried out as illustrated in theflowchart of FIGS. 16A and 16B.

First, the number of strokes in the i-th reproduction object is checkedby analyzing the data in the body part of the data copied in S76 tocount the stroke data pieces, and this value is stored as numStrk in theMEM 14 (S140).

It is determined whether numStrk is greater than 0 (S142) and theprocessor goes to S144 in the case of affirmative determination. Thedetermination of S142 is repetitively carried out until numStrk isdetermined to be not more than 0, i.e., until it is determined that thefollowing processes of S144 to S166 to reproduce a stroke are completedfor the all strokes counted in S140.

In S144 the processor is kept in a sleep state for the time set indiffTime. If the stroke as a currently processed object is the firststroke, diffTime in S112 is the difference between the voice start timeand the stroke start time, stored in S102, whereas diffTime in S130 is 0set in S126.

The strokes to be processed in S144 to S166 are the stroke data countedin S140, and they are selected as an object of the processes in S144 toS166 in order from the earliest stroke, based on comparison of St (inputstart time) in the header part of each stroke data. The stroke data as aprocessed object will be called hereinafter current stroke data.

The processor reads the attributes of the current stroke data (S146),determines a draw time of next stroke data nextDrawTime, and stores itin the MEM 14 (S148). This nextDrawTime is obtained by adding thedifference between St of the current stroke data and St of the nextstroke data to the present time (the present time in the clock functionin the apparatus).

The processor reads the number of points of the current stroke data fromthe MEM 14 and stores it as numPts in the MEM 14 (S150). This number ofpoints is given by the value of n in the body part of the stroke data.

Then the processor reads the start coordinate point (P1) from the MEM 14and stores it as prePts in the MEM 14 (S152). Then the processorinitializes the counter i in the MEM 14 to 1 (S154) and also initializesdiffPtTime to 0 (S156).

It is determined whether i is not more than numPts (S158) and theprocessor goes to S160 in the case of affirmative determination. Thedetermination of S158 is repetitively carried out until it is determinedthat i is larger than numPts, that is, until the reproduction operationof S160 to S166 is completed for the all coordinate points of thecurrent stroke.

Then the processor is kept in a sleep state for the time set asdiffPtTime (S160), draws a straight line connecting the coordinatesstored as prePts with the coordinates of the i-th point (S162), updatesthe data in the MEM 14 to set the difference between the i-th coordinatetime (Pit) and the (i+1)th coordinate time (P(i+1)t) as a value ofdiffPtTime (S164), increments i by one (S166), and returns to S158.

When the determination is negative in S158, the processor goes to S168to decrement numStrk by one and then compares nextDrawTime with thecurrent time in S170. When it is determined in S170 that nextDrawTime islarger than the current time (curTime), the processor updates the valueof diffTime in the MEM 14 to the value obtained by nextDrawTime-curTime(the difference between nextDrawTime determined in S148 and the currenttime) (S172). When it is determined in S170 that nextDrawTime is notgreater than the current time, the processor updates diffTime in the MEM14 to 0 (S174).

By repetitively carrying out the operation of S160 to S166, one strokeis reproduced at the same speed, in the same shape, and at the sameposition as those upon entry. By repetitively carrying out the operationof S144 to S174, the all strokes in one object are reproduced at thesame speed and at the same time intervals as those upon entry thereof.

If the attributes read in S146 are drawn upon drawing of the stroke inS162, it becomes possible to make the user recognize reproduction of thestroke even without erasure of the stroke in S82.

Next described is the data in which a plurality of strokes drawn in thedraw mode are grouped and handled as one object, as describedpreviously, and which is stored as cluster data in the MEM 14.

FIG. 18 illustrates an example in which two objects are drawn in thedraw mode. In this document there are an object 1100 composed of twostrokes to indicate a tree, and an object 1020 composed of thirteenstrokes to indicate the sun. When in the edit mode the vicinity of astroke is clicked by cursor, the whole object including that stroke isselected and a circumscribed rectangle around the selected object isdisplayed based on the data of Ox, Oy, Cx, Cy in the header part of theselected object. This display of the circumscribed rectangle informs theoperator that the object is selected and which strokes belong to theobject. FIG. 18 shows a state in which the cursor 430 is located nearone stroke of the object 1020 and clicked there whereby the object 1020including that stroke is selected and the circumscribed rectangle 1021around that object is displayed.

The processes carried out in S20, including the edit process, movement,cut, copy, paste, clear, smooth, engross, rotate, resize, etc., areeffected on the whole of this selected object.

Now, let us explain the process for moving the selected object by anexample to carry out the movement process for moving the object 1100 ofthe tree toward the object 1020 of the sun according to the flowchart ofFIG. 19.

FIG. 20 illustrates the display screen where the object 1100 is selectedby the cursor 430 and a circumscribed rectangle 1101 is displayed. Whenthe cursor 430 is dragged in this state, amounts of movement of thecursor in the x-axis direction and in the y-axis direction are stored asxOffset and yOffset in the MEM 14 (S230). The cluster of the object tobe moved (the cluster of the object 1100 in FIG. 20) is stored ascurClstr in the MEM 14 (S232). It is determined whether the data ofcurClstr is grouped data or not (S234). When it is determined to begrouped data, the processor adds xOffset or yOffset to rectangularinformation xOrigin or yOrigin, respectively, of all clusters incurClstr (S236). This means that when plural strokes are grouped as anobject like the object 1100, xOffset or yOffset is added to therectangular information of the all strokes, xOrigin or yOrigin,respectively.

If it is determined in S234 that the data of curClstr is not groupeddata, xOffset or yOffset is added to the rectangular information ofcurClstr, xOrigin or yOrigin, respectively (S238). After completion ofthe process of S236 or S238 to add xOffset and yOffset, the informationof the root cluster is updated (S240). When the continuous press on thebutton of the mouse 20 is freed to end the drag, the processordetermines the end of the movement instruction and replaces the datastored in the MEM 14 with the data updated in S236 to S240 as the dataafter movement, whereby the strokes are displayed as moved.

FIG. 22 is a diagram after completion of the movement process, in whichthe display of the object 1100 is updated to the position designated bythe cursor. In the middle of the movement, as shown in FIG. 21, theobject 1100 is displayed as held at the position upon the selectionbefore the instruction of the movement and a stroke 1103 is displayed asmoving following the cursor. This stroke 1103 is a temporary displayduring the movement, which is indicated based on the data in curClstr.However, this display does not reflect the attributes of the stroke andis given by a simple line (for example, the color of the line is blackand the thickness of the line is finest), for easy change of display.

Let us next explain the eraser process in S22 according to the flowchartof FIGS. 23A and 23B.

When the eraser mode button 803 is clicked, the mode is switched to theeraser mode and the mouse cursor is switched to an image of an eraser(an eraser cursor 1000 in an example of the screen in FIG. 24). Inresponse to, a press on the button of mouse 20 or a detection of pendown by the digitizer 24, the eraser cursor is displayed at thatposition and coordinate values thereof are stored as ptErase in the MEM14 (S180). The number of objects in the document is read from the datastored in the MEM 14 and is stored as numObj in the MEM 14 (S182). Thenthe counter i is initialized to 1 (S190). It is then determined whetheri is not more than numObj (S192) and the processor goes to S194 in thecase of affirmative determination. This determination of S192 aboutwhether i is not more than numObj is repetitively carried out until i isdetermined to be more than numObj, that is, until it is determined thatthe determination step of whether the position of the eraser cursor isincluded in an object (S194) has been carried out for the all objectscounted in S182.

In S194 it is determined whether the eraser cursor is included in arectangle of the i-th object, by comparing ptErase with the informationto indicate the circumscribed rectangle in the header part of the i-thobject to determine whether ptErase is included in the circumscribedrectangle. When the eraser cursor is determined to be included, ptEraseis subsequently compared with the information to indicate thecircumscribed rectangle in the header part of each cluster of the i-thobject to extract all clusters including ptErase (S198). Then theprocessor counts the number of cluster data extracted to store thenumber as numClstr in the MEM 14 (S200) and initializes the counter j to1 (S202). Then the processor determines whether j is not more thannumClstr (S204) and goes to S206 in the case of affirmativedetermination. In the case of negative determination, the determinationmeans that the determination of whether the eraser cursor is locatednear the stroke indicated by the cluster (S206) is completed for the allclusters extracted in S198, and thus the processor goes to S196 toincrement i by one and then moves to comparison with another object.

In S206 it is determined whether ptErase is a point near the stroke inthe j-th cluster. This determination is carried out as follows; theprocessor compares distances of straight lines connecting betweenptErase and each coordinate point in the j-th cluster with apredetermined threshold and determines that ptErase is a point close tothe stroke in the j-th cluster if it is closer than the threshold of thepredetermined distance value preliminarily stored in the MEM 14. If thedetermination in S206 is negative, the processor increments j by one inS208 and goes to comparison with the next cluster. If the point isdetermined to be a point near the stroke in S206, the processor goes toS210 to delete the pertinent cluster (the j-th cluster) from the MEM 14and from the area on the display screen.

FIG. 25 illustrates an example in which the thirteen strokes are groupedas an object, as previously described in FIG. 18, and each stroke ishandled as separate cluster data independent from the others. Dottedlines represent circumscribed rectangles of the respective clusters.Supposing the eraser cursor 1000 is located near the stroke 1026, it isdetermined in S206 that the position of the eraser cursor 1000 is apoint near the stroke 1026 and the stroke 1026 is erased in S210. Thescreen in which the stroke 1026 is erased is illustrated in FIG. 26.After one cluster data forming the object is erased, the object 1021 iscomposed of the twelve remaining strokes. Although in FIG. 25 and FIG.26 there are illustrated the frames of the dotted lines to explain therespective clusters, the frames of the dotted lines are not actuallydisplayed in the eraser mode.

The example stated herein was the example in which the object to beprocessed was a single stroke only in the eraser mode, but it should benoted that this is not limited to the eraser mode and that in anotheredit function the extraction and process execution of only a singlestroke can also be performed by carrying out the operation of S204 toS220 during execution of the function with the processed object beingthe single stroke. In this case, it is preliminarily determined for eachof the kinds of the edit modes that the object to be processed should beeither a single stroke or the all strokes belonging to one group, thedetermination is stored in the form of a table in the memory 14, and theobject to be processed is switched between the two modes according todetermination about which object mode is set in the edit modedesignated.

The above described the entry and edit of stroke in the processing ofdocument, but these processes can also be utilized as documentprocessing functions in electronic mail and the like.

In the electronic mail, the processes carried out during preparation ofdocument by the sender are the same as the above processes, and thefollowing describes, referring to the flowchart of FIG. 27, a processfor quoting a received mail when the receiver of the electronic mailincluding the stroke and voice data sends a reply mail.

The processor reads information of an inserting comment from a quotationformat memory area of the MEM 14 (S250), calculates deviation amounts inthe x-direction and in the y-direction in the inserting commentinformation, and stores them as xMove and yMove in the MEM 14 (S252).The x-directional deviation amount is determined as follows; theprocessor reads an inserting comment text to the head of the documentset in the quotation format, calculates an x-directional lengthnecessary for the inserting comment from the number of lines and theformat (the line pitch, character size, etc.) for output of theinserting comment, and defines the value as the x-directional deviationamount. The y-directional deviation amount is determined by calculatinga y-directional length necessary for a quotation symbol from the numberof characters in the quotation symbol at the beginning of line in thequotation format and the format for output of the quotation symbol(character pitch, character size, etc.) and defining that value as they-directional deviation amount. The quotation symbol is a characterstring that is repetitively inserted at the beginning of each line in aquoted text, and the inserting comment text is a character string thatis inserted at the head of the quoted text. The contents, characternumbers, and line numbers of the inserting comment text and quotationsymbol can be arbitrarily set by the user. By storing the setinformation in the memory 14, adequate deviation amounts can becalculated upon the quotation process.

The processor reads the original ink data comprised of the stroke dataand voice data of the quoted mail from the memory area of MEM 14 storingthe received data, and copies it onto the working area in the MEM 14 tostore it (S254). Here, the term “ink data” means a stroke data and adata by which the stroke data is accompanied. Then the processor setsthe value of xMove, stored in the MEM 14, as a value of xOffset and thevalue of yMove as a value of yOffset and stores them in the MEM 14(S256).

Then the processor invokes the movement routine for the ink data copiedin S254 (S258) and executes the movement routine (S260). The movementroutine is the one as illustrated in the flowchart of FIG. 19.

Then the processor draws the ink data after movement in a memory fordisplay and displays it on the screen (S262).

In the flowchart of FIG. 27 the movement of the ink data (stroke dataand voice data) was described, but the same process as the insertion ofthe inserting comment text and quotation symbol in the process of thetext edit is carried out for the text included in the received mail.Therefore, the relative positional relation between the text dataincluded in the received mail and the ink data is not changed betweenbefore and after the quotation process.

FIG. 28 illustrates a receive screen of an electronic mail. The contentsof the mail are comprised of the text “Please register for attendance atthe today's meeting.” (2000), the stroke 2100 to indicate “attendance”and “absence,” the text “PS. The meeting will be held in meeting roomA.” (2001), and the signature 2002 of the originator of the mail. When areply to this mail is designated, the process of FIG. 27 is carried outto quote the contents of the received mail and to display the screen inwhich the inserting comment 2200 and the quotation symbol 2201 areinserted at the head and at the beginning of each line as shown in FIG.29.

The execution of the quotation process shown in the flowchart of FIG. 27causes no change between the relative positional relation (FIG. 28)between the text data and the ink data at the time of reception of themail and the relative positional relation (FIG. 29) between the textdata and the ink data at the time of reply to the mail.

1. A computer-executable processing method comprising: storing areceived mail document including text data and ink data, an ink imagebeing dynamically reproduced from the ink data and overlaid on a textimage reproduced from the text data when the mail document isreproduced, the ink data including coordinate information of each inkimage and time information comprising a reproduction time for each inkimage, a reproduction position of the ink image being defined by thecoordinate information on reference coordinate axes of the received maildocument, and a reproduction speed of the ink image being defined by thetime information, the text data is non-ink data; inserting a characterstring to text data of a new document when a new document quoting thereceived mail document is prepared; calculating a coordinate shiftamount of the dynamic reproduction position of the ink image accordingto a new text image reproduced from the text data to which the characterstring was inserted; and outputting, as the new document, the ink imagewhich is overlaid on the new text image reproduced from the text data towhich the character string was inserted, the dynamic reproduction of theink image being executed based on the calculated coordinate shiftamount, the coordinate information and the time information.
 2. Theinformation processing method according to claim 1, wherein the ink datacomprises locus information to define the output position by coordinatevalues.
 3. The information processing method according to claim 1,wherein said character string to be inserted is a quotation symbol. 4.The information processing method according to claim 1, wherein saidcharacter string to be inserted is an inserting comment text.
 5. Theinformation processing method according to claim 1, wherein saidcharacter string to be inserted is a character string that can beedited.
 6. The information processing method according to claim 1,wherein said shift amount is length information.
 7. The informationprocessing method according to claim 1, wherein the new output documentis carried out by setting said shift amount as an offset value of saidreceived mail document.
 8. The information processing method accordingto claim 1, wherein said shift amount is calculated according to anumber of lines of the character string to be inserted and a line pitchof the document format.
 9. The information processing method accordingto claim 1, wherein said shift amount is calculated according to anumber of lines and a number of characters of the character string to beinserted and according to a line pitch and a character pitch of thedocument format.
 10. A computer-executable processing method comprising:storing document information comprising locus information and textinformation, a locus image being dynamically reproduced from the locusinformation and overlaid on a text image being reproduced from the textinformation when the document is reproduced, the locus informationincluding coordinate information of each locus image and timeinformation comprising a reproduction time for each locus image, areproduction position of the locus image being defined by the coordinateinformation on reference coordinate axes of the document information,and a reproduction speed of the locus image being defined by the timeinformation, the text data is non-ink data; editing said textinformation; calculating a coordinate shift amount of the dynamicreproduction position of the locus image according to a new text imagereproduced from the edited text information; and outputting the locusimage which is overlaid on the new text image reproduced from the editedtext information, the dynamic reproduction of the locus image beingexecuted based on the calculated coordinate shift amount, the coordinateinformation and the time information.
 11. The information processingmethod according to claim 10, wherein the calculated shift amount is adifference between a position of the text image upon output thereofwithout the editing and a position of the text image upon output thereofafter the editing.
 12. The information processing method according toclaim 10, wherein said editing is insertion of a character string. 13.The information processing method according to claim 10, wherein theshift amount is coordinate data.
 14. An information processing apparatuscomprising: received mail storing means for storing a received maildocument including text data and ink data, an ink image beingdynamically reproduced from the ink data and overlaid on a text imagereproduced from the text data when the mail document is reproduced, theink data including coordinate information of each ink image and timeinformation comprising a reproduction time for each ink image, areproduction position of the ink image being defined by the coordinateinformation on reference coordinate axes of the received mail document,and a reproduction speed of the ink image being defined by the timeinformation, the text data is non-ink data; insertion means forinserting a character string to text data of a new document when a newdocument quoting the received mail document is prepared; shift amountcalculating means for calculating a coordinate shift amount of thedynamic reproduction position of the ink image according to a new textimage reproduced from the text data to which the character string wasinserted; and output means for outputting, as the new document, the inkimage which is overlaid on the new text image reproduced from the textdata to which the character string was inserted, the dynamicreproduction of the ink image being executed based on the calculatedcoordinate shift amount, the coordinate information and the timeinformation.
 15. The information processing apparatus according to claim14, wherein the ink data comprises locus information to define theoutput position by coordinate values.
 16. The information processingapparatus according to claim 14, wherein said character string to beinserted is a quotation symbol.
 17. The information processing apparatusaccording to claim 14, wherein said character string to be inserted isan inserting comment text.
 18. The information processing apparatusaccording to claim 14, wherein said character string to be inserted is acharacter string that can be edited.
 19. The information processingapparatus according to claim 14, wherein said shift amount is lengthinformation.
 20. The information processing apparatus according to claim14, wherein the new output document is carried out by setting said shiftamount as an offset value of said received mail document.
 21. Theinformation processing apparatus according to claim 14, wherein saidshift amount is calculated according to a number of lines of thecharacter string to be inserted and a line pitch of the document format.22. The information processing apparatus according to claim 14, whereinsaid shift amount is calculated according to a number of lines and anumber of characters of the character string to be inserted andaccording to a line pitch and a character pitch of the document format.23. The information processing apparatus according to claim 14, whereinsaid output means is an ink jet printer.
 24. The information processingapparatus according to claim 14, wherein said output means is a printer.25. The information processing apparatus according to claim 14, whereinsaid output means is a display device.
 26. An information processingapparatus comprising: storage means for storing document informationcomprising locus information and text information, a locus image beingdynamically reproduced from the locus information and overlaid on a textimage being reproduced from the text information when the document isreproduced, the locus information including coordinate information ofeach locus image and time information comprising a reproduction time foreach locus image, a reproduction position of the locus image beingdefined by the coordinate information on reference coordinate axes ofthe document information, and a reproduction speed of the locus imagebeing defined by the time information, the text data is non-ink data;text edit means for editing said text information; shift amountcalculating means for generating a coordinate shift amount of thedynamic reproduction position of the locus image according to a new textimage reproduced from the edited text information; and output means foroutputting the locus image which is overlaid on the new text imagereproduced from the edited text information, the dynamic reproduction ofthe locus image being executed based on the calculated coordinate shiftamount, the coordinate information and the time information.
 27. Theinformation processing apparatus according to claim 26, wherein thecalculated shift amount is a difference between a position of the textimage upon output thereof without the editing and a position of the textimage upon output thereof after the editing.
 28. The informationprocessing apparatus according to claim 26, wherein said editing isinsertion of a character string.
 29. The information processingapparatus according to claim 26, wherein the shift amount is coordinatedata.
 30. A storage medium for storing computer-executable process stepsfor an information processing method, said storage medium storing: codefor storing a received mail document including text data and ink data,an ink image being dynamically reproduced from the ink data and overlaidon a text image reproduced from the text data when the mail document isreproduced, the ink data including coordinate information of each inkimage and time information comprising a reproduction time for each inkimage, a reproduction position of the ink image being defined by thecoordinate information on reference coordinate axes of the received maildocument, and a reproduction speed of the ink image being defined by thetime information, the text data is non-ink data; code for inserting acharacter string to text data of a new document when a new documentquoting the received mail document is prepared; code for calculating acoordinate shift amount of the dynamic reproduction position of the inkimage according to a new text image reproduced from the text data towhich the character string was inserted; and code for outputting, as thenew document, the ink image which is overlaid on the new text imagereproduced from the text data to which the character string wasinserted, the dynamic reproduction of the ink image being executed basedon said calculated coordinate shift amount, the coordinate informationand the time information.
 31. A storage medium for storingcomputer-executable process steps for an information processing method,said storage medium storing: code for storing document informationcomprising locus information and text information, a locus image beingdynamically reproduced from the locus information and overlaid on a textimage being reproduced from the text information when the document isreproduced, the locus information including coordinate information ofeach locus image and time information comprising a reproduction time foreach locus image, a reproduction position of the locus image beingdefined by the coordinate information on reference coordinate axes ofthe document information, and a reproduction speed of the locus imagebeing defined by the time information, the text data is non-ink data;code for editing said text information; code for calculating acoordinate shift amount of the dynamic reproduction position of thelocus image according to a new text image reproduced from the editedtext information; and code for outputting the locus image which isoverlaid on the new text image reproduced from the edited textinformation, the dynamic reproduction of the locus image being executedbased on the calculated coordinate shift amount, the coordinateinformation and the time information.